{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(code-where)=\n",
    "# Options for Writing Code\n",
    "\n",
    "In this chapter, we'll look at the different ways you can both write and run code. This is something that can be very confusing if you're just getting into coding.\n",
    "\n",
    "There are different ways to write (and run) code that suit different needs. For example, for creating a reproducible pipeline of tasks or writing production-grade software, you might opt for a script——a file that is mostly code. But for sending instructions to a colleague or exploring a narrative, you might choose to write your code in a notebook because it can present text and code together more naturally than a script can.\n",
    "\n",
    "We already met some ways to write and run code in the subsequent chapters. Here, we'll be a bit more systematic so that, by the end of the chapter, you'll be comfortable writing code in both scripts and notebooks. For advanced users, there's also information on how to write code with markdown, using markdown files that contain executable code chunks. Scripts and notebooks are by far the most popular ways to write code though.\n",
    "\n",
    "Let's start with some definitions.\n",
    "\n",
    "- IDE, or **integrated development environment**: this is the application that you write code of all different kinds in (scripts, notebooks, markdown). This book recommends Visual Studio Code as an IDE. It supports many languages. Making an analogy with writing documents, VS Code is to programming languages what a word processor is for actual written languages. JupyterLab is another IDE, but one which is geared towards the use of notebooks.\n",
    "- the **interpreter**: this is the programming language (eg Python) that has to be installed separately onto your computer. Its what takes your written commands and turns them into actions. VS Code and other IDEs will use whatever interpreters they can find on your computer and use them to execute the code you've written.\n",
    "- **scripts**: these are files that almost exclusively contain code. They can be edited and run in an IDE. Python scripts always have the file extension `.py`.\n",
    "- **notebooks**: aka Jupyter Notebooks, these are files that can contain code and text in different blocks called \"cells\". They can be edited, and the code parts can be run, in an IDE. Jupyter Notebooks always have the file extension `.ipynb`. Notebooks can be export to other formats, like word documents, HTML pages, PDFs, and even slides!\n",
    "- **the terminal**: this is the text interface that you use to send instructions to your computer's operating system. It's typically what you use to install new packages, for example with commands like `pip install packagename`. Although your computer will come with a separate terminal application too, you can open a terminal in VS Code by clicking on Terminal > New Terminal at the top of your VS Code window.\n",
    "- **markdown**: this is a lightweight language that turns simple text commands into professional looking documents. It's widely used by people who code. It's also what's used for the text cells in Jupyter Notebooks. When not in a notebook, files containing markdown always have the extension `.md`. With the Visual Studio Code markdown extensions installed, you can right-click within a Markdown file in VS Code and then select *Markdown Preview Enhanced* to see how the rendered document will look. The difference between HTML code and the same website viewed in a browser like Chrome is a good analogy for the difference between what you see in a `.md` file and what you see in the preview of that markdown file. You can find out more about markdown in the chapter {ref}`wrkflow-markdown`.\n",
    "\n",
    "Let's now turn to all of the different ways you can write code in a fully-featured integrated development environment like Visual Studio Code. They each have pros and cons, and you're likely to want to use them at different times. The table below sets out all of the different ways you can write, and execute, code.\n",
    "\n",
    "If you're looking for a typical workflow though, this book recommends working with scripts (files that end in `.py`) and the VS Code interactive window. Remember, if you're working with a `.py` file, you can always open the Visual Studio Code interactive window by right-clicking somewhere within the script and selecting 'Run in interactive window'.\n",
    "\n",
    "\n",
    "| What | How to use  | Prerequisites | Pros | Cons\n",
    "|------|----------------|---------------|---------------|---------------|\n",
    "| Script, eg `script.py`   |   1. Run `python script.py` on the command line; or 2. 'Run in interactive window' in an integrated development environment (IDE)         | 1. Python installation,  2. Python installation + an IDE with Python support, eg Visual Studio Code. | Can be run all-in-one or step-by-step as needed. Very powerful tools available to aid coding in scripts. De facto standard for production-quality code. Can be imported by other scripts. Version control friendly. | Not very good if you want to have lots of text alongside code.\n",
    "| Jupyter Notebook, eg `notebook.ipynb`   | 1.  In the same folder as the file, run `jupyter lab` on the command line, which will open a browser window, then follow the options to open the relevant file; or 2. Open the file with Visual Studio Code.     |    1. Python installation with jupyter lab installed (install by running `pip install jupyterlab` on the command line), 2. Everything in 1 + Visual Studio Code and the VS Code Jupyter extension.  | Code and text can alternate in the same document. Rich outputs of code can be integrated into document. Can export to PDF, HTML, and more, with control over whether code inputs/outputs are shown. Can be run all-in-one or step-by-step as needed. | Fussy to use with version control. Code and text cannot be mixed in same 'cell'. Not easy to import in other code files.\n",
    "| Markdown with executable code chunks, eg `markdown_script.md`   | To produce markdown output, use `codebraid pandoc --from markdown --to markdown markdown_script.md` on the command line. Other output types available. | Installations of Python, codebraid (a Python package), and pandoc (an application). | Allows for true mixing of text and code. Can export to wide variety of other formats, such as PDF and HTML, with control over whether code inputs/outputs are shown. Version control friendly. | Must be run all-in-one so cannot see outputs of individual code-chunks as you go. Cannot be imported by other code files.\n",
    "\n",
    "Some of the options above make use of the command line, a way to issue text-based instructions to your computer. Remember, the command line (aka the terminal) can be accessed via the Terminal app on Mac, the Command Prompt app on Windows, or <kbd>ctrl</kbd> + <kbd>alt</kbd> + <kbd>t</kbd> on Linux. To open up the command line within Visual Studio Code, you can use the keyboard shortcut <kbd>⌃</kbd> + <kbd>\\`</kbd> (on Mac) or \n",
    "<kbd>ctrl</kbd> + <kbd>\\`</kbd> (Windows/Linux), or click \"View > Terminal\".\n",
    "\n",
    "```{tip}\n",
    "The command line, or terminal, is covered in more detail in a subsequent chapter.\n",
    "```\n",
    "\n",
    "Now let's look at each of these ways to run code in more detail using a common example: Hello World!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scripts\n",
    "\n",
    "We already met scripts, but let's have a recap. Create a new file in Visual Studio Code called `hello_world.py`. In the Visual Studio Code editor, add a single line to the file:\n",
    "\n",
    "```python\n",
    "print('Hello World!')\n",
    "```\n",
    "\n",
    "Save the file. Right-click and, to run the script, you can either use 'Run current file in interactive window', or 'Run current file in terminal', or 'Run selection/line in interactive window'. These are two different methods of running the script: in the IDE (VS Code in this case) or in the command line.\n",
    "\n",
    "A typical workflow would be selecting some lines within a script, and then hitting 'Run selection/line in interactive window' or using the keyboard shortcut of <kbd>shift</kbd> + <kbd>enter</kbd>.\n",
    "\n",
    "As an alternative for the latter, you can open up the command line yourself and run\n",
    "\n",
    "```bash\n",
    "python hello_world.py\n",
    "```\n",
    "\n",
    "which will execute the script."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jupyter Notebooks\n",
    "\n",
    "Jupyter Notebooks are for experimentation, tinkering, and keeping text and code together. They are the lab books of the coding world. The name, 'Jupyter', is a reference to the three original languages supported by Jupyter, which are Julia, Python, and R, and to Galileo's notebooks recording the discovery of the moons of Jupiter. Jupyter notebooks now support a vast number of languages beyond the original three, including Ruby, Haskell, Go, Scala, Octave, Java, and [more](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels).\n",
    "\n",
    "To get started with Jupyter Notebooks, you'll need to have a Python installation and to run `pip install jupyterlab` on the command line.\n",
    "\n",
    "If you get stuck with this tutorial, there's a more in-depth VS Code and Jupyter tutorial available [here](https://code.visualstudio.com/docs/python/jupyter-support).\n",
    "\n",
    "Create a new file in Visual Studio Code and save it as `hello_world.ipynb`. Close the file and re-open it. The notebook interface should automatically load and you'll see options to create cells with plus signs labelled 'Code' and 'Markdown'. A cell is an independent chunk of either code or text. Text cells have markdown in them. Create a markdown cell containing the following:\n",
    "\n",
    "```markdown\n",
    "# This is a title\n",
    "\n",
    "## This is a subtitle\n",
    "\n",
    "This notebook demonstrates printing 'hello world!' to screen.\n",
    "```\n",
    "\n",
    "Now, for the next cell, choose code and write:\n",
    "\n",
    "```python\n",
    "print('hello world!')\n",
    "```\n",
    "\n",
    "To run the notebook, you can choose to run all cells (usually a double play button at the top of the notebook page) or just each cell at a time (a play button beside a cell). 'Running' a markdown cell will render the markdown in display mode; running a code cell will execute it and insert the output below. When you play the code cell, you should see the 'hello world!' message appear.\n",
    "\n",
    "Although Visual Studio Code is very convenient for using notebooks, Jupyter Lab is an alternative that has quite a few more options, for example exporting to HTML, PDF, and markdown with code outputs included. To try this out, open a terminal window in the folder that contains your '.ipynb' file. Type `jupyter lab`. This should open a browser window with some options such as Python and any other languages you might have installed. Choose Python and then select `hello_world.ipynb`. If you run the notebook, you can then use the 'File->Export As' options to export the text, code, and code outputs (including figures, if there were any) as HTML, LaTeX, PDF, slides, markdown, and more-though some of these depend on other applications and settings. Once you've finished running your Jupyter Lab session, you can shut it down in the terminal with `ctrl+c`.\n",
    "\n",
    "Jupyter Notebooks are versatile and popular for early exploration of ideas, especially in fields like data science. This entire book is written in a combination of Jupyter Notebooks and executable markdown (more on that in a moment). Jupyter Notebooks can easily be run in the cloud using a browser too (via Binder or Google Colab) without any prior installation. Although it's not got very much code in, the page you're reading now can be loaded into Google Colab as a Jupyter Notebook by clicking 'Colab' under the rocket icon at the top of the page.\n",
    "\n",
    "One really nice feature of Jupyter Notebooks is that you can export them without the code inputs, but just showing the text and the code outputs. This is great for sending reports to colleagues who are not interested in the details of your code. Say you have a notebook called `notebook.ipynb`. You can turn this into a HTML report that doesn't have any code inputs by running the following line in the terminal (in the same directory as `notebook.ipynb`):\n",
    "\n",
    "```bash\n",
    "jupyter nbconvert --no-input --no-prompt notebook.ipynb\n",
    "```\n",
    "\n",
    "Finally, you can try a Jupyter Notebook without installing anything online at [https://jupyter.org/try](https://jupyter.org/try). Click on Try Classic Notebook for a tutorial.\n",
    "\n",
    "### Tips when using Jupyter Notebooks\n",
    "\n",
    "- Version control: if you are using verion control, be wary of saving the outputs of Jupyter Notebooks when you only want to save code. Most IDEs that support Jupyter Notebooks have a clear outputs option. You can also automate this as a *pre-commit git hook* (if you don't know what that is, don't worry). You could also pair your notebook to a script or markdown file (covered in the next section). Outputs or not, Jupyter Notebooks will render on github.\n",
    "\n",
    "- Terminal commands: these can be run from inside a Jupyter Notebook by placing a `!` in front of the command and executing the cell. For example, `!ls` gives the directory the notebook is in. You can also `!pip install` and `!conda install` in this way. This works in Google Colab notebooks too.\n",
    "\n",
    "- Magic commands: statements that begin with `%` are magic commands. `%whos` displays information about defined variables. `%run script.py` runs a script called `script.py`. `%timeit` times how long the cell takes to execute. Finally, you can see many more magic commands using  `%quickref`.\n",
    "\n",
    "- Notebook cells can be executed in any sequence you choose. But if you're planning to share your notebook or use it again for yourself, it's good practice to check that its cells do what you want *when run in sequence*, from top to bottom.\n",
    "\n",
    "- There are tons of extensions to Jupyter Notebooks; you can find a list [here](https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions.html). Of particular note is [ipywidgets](https://ipywidgets.readthedocs.io/en/stable/index.html), which adds interactivity.\n",
    "\n",
    "- Get help info on a command by running it but with `?` appended to end."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Markdown with executable code chunks\n",
    "\n",
    "```{warning}\n",
    "This is an advanced topic; feel free to skip it.\n",
    "```\n",
    "\n",
    "This is by far the least common way of coding, though it has gained popularity in recent years. \n",
    "\n",
    "When you have much more text and code, even using Jupyter Notebooks can feel a bit onerous and, historically, editing the text in a notebook was a bit tedious - especially if you wanted to move cells around a lot. Markdown makes for a much more pleasant writing experience. \n",
    "\n",
    "*Markdown* is, like HTML, a mark-up language: a way of specifying rich text (such as links, including an image, displaying bullets, italics) using simple text commands. What this means is that by using a few extra symbols, Markdown creates a document with rich features. For example, here is the markdown syntax for showing different text styles:\n",
    "\n",
    "```markdown\n",
    "*this is displayed in italics*\n",
    "**this is displayed in bold**\n",
    "```\n",
    "\n",
    "Markdown supports LaTeX equations too:\n",
    "\n",
    "```markdown\n",
    "$$u'(c_{t})=\\beta(1+r_{t+1})u'(c_{t+1})$$\n",
    "```\n",
    "\n",
    "produces\n",
    "\n",
    "$$u'(c_{t})=\\beta(1+r_{t+1})u'(c_{t+1})$$\n",
    "\n",
    "Because all you need to write markdown is a simple text editor, it has found huge popularity in the coding and academic communities. Sites like github use it. Viewing markdown is less easy, but the **markdown-all-in-one** extension of VS Code can show a preview of how it will look. If you open a markdown file in VS Code, just right click and select preview.\n",
    "\n",
    "Regular code chunks (ones that are *not* executed) are written with three backticks, followed by the name of the language, followed by the code, followed by another three backticks. So to express a non-executable Python code chunk (ie a chunk that is displayed only) it's\n",
    "\n",
    "````\n",
    "```python\n",
    "import pandas as pd\n",
    "```\n",
    "````\n",
    "for example. And this gets rendered as\n",
    "\n",
    "```python\n",
    "import pandas as pd\n",
    "```\n",
    "\n",
    "Adding *executable code chunks* to markdown allows you to include code chunks *and* the outputs they create when they are run *or* just the outputs (with the code hidden). This is useful when you want to write something that just has a little bit of code in but is mostly text. Or when you want to create documents or slides that update automatically by displaying the outputs of code run on updating inputs. Markdown with executable code chunks allows you to do all of these.\n",
    "\n",
    "The magic package that weaves together code and markdown is called [**codebraid**](https://github.com/gpoore/codebraid). **codebraid** can take markdown documents, execute any code in them, and incorporate the outputs of that code in a final version of the document. It has built-in code execution that currently supports Python 3.5+, Julia, Rust, R, Bash, JavaScript, and SageMath. It can mix and match languages within the same document too.\n",
    "\n",
    "Let's now try an example with an *executable code chunk*. For this, you will need to have installed the [**codebraid**](https://github.com/gpoore/codebraid) Python package and [**pandoc**](https://pandoc.org/installing.html), an application. Pandoc is a universal document converter. Codebraid 'braids' together code and text.\n",
    "\n",
    "Create a new markdown file called 'hello_world_raw.md' and add this:\n",
    "\n",
    "````\n",
    "---\n",
    "title: \"Codebraid Hello World! example\"\n",
    "---\n",
    "\n",
    "# This is a title\n",
    "\n",
    "## This is a subtitle\n",
    "\n",
    "This executable markdown demonstrates printing 'hello world!' to screen.\n",
    "\n",
    "```{.python .cb.nb jupyter_kernel=python3}\n",
    "print('Hello World!')\n",
    "```\n",
    "\n",
    "Now we can have more text afterwards, and a code chunk that hides itself.\n",
    "\n",
    "```{.python .cb.run}\n",
    "print(\"Hello World! But don't show the code.\")\n",
    "```\n",
    "\n",
    "````\n",
    "\n",
    "In the 'raw' input markdown file, `.python` tells codebraid that a code chunk is in Python, `jupyter_kernel=python3` tells codebraid what installation of Jupyter Notebooks to use, while `.cb.nb` tells a Python chunk to run like a notebook (showing both inputs and outputs). `.cb.run` is an instruction to just run the code and insert the output (doesn't show code input). If you're unsure what your installation of Jupyter is called, you can see a list by running `jupyter kernelspec list` on the command line.\n",
    "\n",
    "Once you have saved the file, using the command line, run\n",
    "\n",
    "```bash\n",
    "codebraid pandoc hello_world_raw.md -o hello_world.md --overwrite\n",
    "```\n",
    "\n",
    "and then open up 'hello_world.md'. You should see the following:\n",
    "\n",
    "````\n",
    "# This is a title\n",
    "\n",
    "## This is a subtitle\n",
    "\n",
    "This executable markdown demonstrates printing 'hello world!' to screen.\n",
    "\n",
    "``` {.python .numberLines startFrom=\"1\"}\n",
    "print('Hello World!')\n",
    "```\n",
    "\n",
    "``` {.stdout}\n",
    "Hello World!\n",
    "```\n",
    "\n",
    "Now we can have more text afterwards, and a code chunk that hides itself.\n",
    "\n",
    "Hello World! But don't show the code.\n",
    "````\n",
    "\n",
    "The code chunks in the markdown have been executed and the relevant text has appeared in the *output* markdown file. The same rule applies to rich content like figures too. Use the preview of the rendered file to see what it looks like (right click in VS Code, then Preview).\n",
    "\n",
    "The powerful combination of markdown and codebraid doesn't just allow you to write markdown and output executed markdown: you can output a wide range of formats, including Word documents, PDFs (papers and beamer presentations), Powerpoint presentations, HTML slides, HTML pages. To switch to a different output format, the general syntax is `codebraid pandoc hello_world_raw.md -o hello_world.extension --overwrite` where 'extension' is the file extension.\n",
    "\n",
    "```{admonition} Challenge\n",
    "Try writing an executable code chunk that produces a figure in the **codebraid** output; then try doing the same but without showing the code input and outputting to PDF format.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Paired notebook and script or markdown\n",
    "\n",
    "```{warning}\n",
    "This is an extremely advanced topic; feel free to skip it.\n",
    "```\n",
    "\n",
    "Can't decide between notebook on the one hand and script or executable markdown on the other? You can have both! Another tool, [**jupytext**](https://jupytext.readthedocs.io/en/latest/index.html), allows you to pair a notebook with either a script (.py file) or an executable markdown file (.md). **jupytext** supports Julia, Python and R.\n",
    "\n",
    "If you install **jupytext**, the easiest way to use it is via Jupyter Lab, where it will appear as an option when you're working on a notebook. Type \"Pair notebook\" in the Command Palette and you will see some options appear.\n",
    "\n",
    "Pairing a notebook with one of these other formats means that whatever you write in your notebook will be reflected in the other format, and vice versa.\n",
    "\n",
    "So should you pair your notebook to a script or a markdown file? If your notebook is mostly made of code, you will probably prefer to save it as a script with the occasional bit of text in. In that case, you have two options for how the text will appear in your Python script, 'percent', which is preceeded by `# %%`, or 'light', which means preceeded only by `#`.  If your notebook contains more text than code, you may want to pair your notebook to a Markdown document, and there are options for that too.\n",
    "\n",
    "Of course, **jupytext** comes with a bunch of commands you can use in the terminal to convert between notebooks, markdown files, and scripts - really handy if you decide you want to change midway through a project.\n",
    "\n",
    "You can find out much more on the [**jupytext**](https://jupytext.readthedocs.io/en/latest/index.html) website."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "c4570b151692b3082981c89d172815ada9960dee4eb0bedb37dc10c95601d3bd"
  },
  "kernelspec": {
   "display_name": "Python 3.8.12 64-bit ('codeforecon': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
